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Why virtual creatures matter 


Tired of training neural networks? Try optimizing virtual creatures instead. 


ie gue. 


An evolved, galloping creature. Adapted from ref. *, ACM. 


he first virtual creatures were 

evolved by Karl Sims in the early 

1990s': morphology and control of 
autonomous agents were co-optimized so 
that they ran, jumped, swam, performed 
phototaxis and fought head-to-head for 
resources in a virtual world that followed 
(more or less) the laws of classical 
mechanics. The creatures were relatively 
simple, composed of just a handful of 
jointed, rigid components, but their 
behaviour was surprisingly rich and lifelike’. 

Karl Sims’s experiments started with 
an objective, such as locomotion, and a 
population of randomly assembled creatures. 
Although it was unlikely that any randomly 
assembled creature would fully satisfy the 
objective, by replacing the worst-performing 
designs with slightly and randomly modified 
copies of the better ones, the population 
made incremental progress, generation by 
generation. It was the survival of the fittest; 
or, in the case of locomotion: the fastest. 

In 2000, Lipson and Pollack’ transferred 
similarly evolved creatures from simulation 
to reality with a technology just then 
emerging: 3D printing. Robot designs were 
rapidly and safely prototyped as virtual 
creatures, discarding the truly awful or 
dangerous designs before testing them in 
reality. The result were ‘robotic lifeforms that 
were designed, optimized and built, end-to- 
end, with almost no human intervention. 

Since then, despite many attempts to 
improve on this seminal work, and despite 
vast increases in computational power, 
there has been relatively little progress 
scaling the complexity, competence and 
usefulness of virtual creatures and their 
3D-printed equivalents. 

This stagnancy was ended in 2013 with 
an important advance by Cheney et al.* — 
winners of the inaugural Virtual Creatures 
Competition — which introduced the 
evolution of soft robots composed of 
thousands of elastic voxels (pictured). This 
work became highly influential, and for 
good reason. 

Soft robots provide a unique opportunity 
for virtual creatures research: soft structures 
can act as shock absorbers, energy stores 
and even power amplifiers (catapults). 
However, they are extremely non-intuitive 
to design and control due to low mechanical 
impedance (force applied to one part of the 
robot can propagate in unanticipated ways 
throughout its body), and because they have 
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much greater morphological possibility: they 
can change shape and material properties, as 
well as their control policies. 

By instantiating soft machines in a virtual 
world, it is possible to rapidly and cheaply 
evaluate many candidate designs in parallel. 
This makes exploration of the design space 
much more efficient than if each proposed 
design needed to be built in reality, and 
permits us to use optimization algorithms to 
automatically discover and refine a robot's 
shape, material properties and controllers 
contemporaneously, thus leading to entirely 
novel solutions outside the purview of 
human intuition’. 

However, the goal of designing virtual 
creatures is not necessarily to design the best 
soft robot. The goal can also be to engage 
our imagination and creativity — to expand 
our conception of life and intelligence, and 
how these things might materialize. 

Indeed, Sims, who was primarily interested 
in computer graphics, viewed his creatures 
as artistic expressions, but others (the author 
of this essay included) saw his work as an 
exciting new paradigm in the science of AI. 

Since its inception, the Virtual Creatures 
Competition has been affiliated with the 
Genetic and Evolutionary Computation 
Conference (GECCO), a machine learning 
conference inspired by evolution and ecology 
rather than learning and neuroscience. This 
is because morphological optimization 
proper — changing the number and 
placement of mechanical degrees of freedom, 
not merely tuning the parameters of a 
predefined structure — has thus far only been 
demonstrated through evolutionary design. 

The winners of this year’s competition, 
Deepak Pathak and colleagues’, have shattered 
this assumption by demonstrating how the 
morphology and controller of a machine 
can be co-optimized using a gradient-based 
learning algorithm, without evolution. Instead 
of evolving a monolithic morphology, 

Pathak et al. optimize a swarm of elemental 
agents — autonomous ‘limbs’ — that, 


in addition to actuating, can choose at every 
timestep to either attach to their nearest 
neighbour (forming an aggregate, symbiotic 
machine with a shared reward function) or 
detach, reconfigure and test a new design. 
Thus morphology is controlled by the same 
policy that coordinates behaviour. 

What comes next for virtual creatures 
research may be to distil and combine the 
most useful properties of evolutionary and 
gradient-based algorithms, and discard the 
rest. However, this will require some finesse. 
More powerful optimization, without 
recourse to testing in the real world, might 
result in overfit virtual creatures forever 
trapped inside the computer. 

But even completely virtual life can hold 
real value. And this is precisely where virtual 
creatures depart from the current paradigm 
of most machine learning research: virtual 
creatures are not constrained by their 
applicability. They are interesting objects of 
scientific investigation in their own right. 
More than that, they have the potential to be 
as beautiful and complex as life itself. o 
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